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1.0  SUMMARY 


United  States  Air  Force  MQ-1  Predator  remotely  piloted  aircraft  pilots  are  critical  assets 
in  a  challenging,  high-risk,  and  rapidly  evolving  career  field.  However,  the  Air  Force  has  had 
difficulties  over  the  past  decade  recruiting  candidates  with  the  “right  stuff”  and  filling  Predator 
pilot  vacancies.  The  purpose  of  this  study  was  to  evaluate  computer-based  intelligence  and 
neuropsychological  testing  on  training  candidates  (n  =  235)  obtained  during  medical  flight 
screening  that  distinguishes  elite  vs.  adequate  MQ-1  qualification  training  performance. 

Findings  reveal  a  combination  of  cognitive  aptitudes  that  distinguish  elite  levels  of  training 
performance  among  those  selected  for  this  career  field.  Results  may  be  utilized  to  improve 
personnel  selection  and  aeromedical  screening  procedures  for  identifying  suitable  candidates  for 
this  high-risk,  high-demand  occupation. 

2.0  INTRODUCTION 

United  States  Air  Force  (USAF)  MQ-1  Predator  remotely  piloted  aircraft  (RPA)  pilots 
are  in  a  high-risk  profession  engaged  in  a  wide  range  of  reconnaissance  and  weapons-strike 
operations  where  mistakes  in  the  perfonnance  of  their  flying  duties  can  lead  to  significant  costs 
in  terms  of  human  life,  national  security,  and  foreign  relations.  Each  year  there  are  several 
aviation-related  mishaps  in  training  and  operational  missions  where  human  error  is  considered  a 
causal  factor.  As  a  result  of  the  high-risk  nature  of  flying  an  RPA  and  the  evidence  of  a  pilot’s 
psychological  disposition  as  crucial  for  safe  and  effective  flying,  the  intellectual  and 
neuropsychological  assessment  of  USAF  RPA  pilots  is  a  critical  part  of  selection  and 
aeromedical  screening  procedures. 

USAF  Predator  RPA  pilots  have  served  a  variety  of  roles  in  providing  real-time 
information  to  commanders  for  tracking  enemy  movements  and  assets,  safeguarding  convoys, 
directing  and  protecting  ground  forces,  locating  and  destroying  weapons  caches,  identifying 
fixed  and  moving  targets,  catching  insurgents  planting  roadside  bombs,  tracking  and/or 
eliminating  enemy  combatants,  augmenting  manned-strike  missions,  surveying  post-strike  battle 
damage,  as  well  as  providing  close  air  support  and  precision  weapons  strikes  [1].  Asa  result,  the 
global  demand  for  Predator  operations  has  increased  significantly  and  military  agencies  are 
having  difficulty  recruiting  and  filling  the  increasing  number  of  RPA  pilot  vacancies  [2,3]. 

Although  aspects  of  RPA  operations  are  highly  automated,  such  pilots  perform  a  wide 
range  of  manual  and  computer-based  tasks  to  actively  and/or  passively  control,  maneuver,  and 
fly  the  aircraft  during  planned,  unplanned,  and  critical  in-flight  situations.  These  tasks  require 
pilots  to  engage  in  complex  decision  making,  to  include  actively  analyzing  and  responding  to 
large  amounts  of  simultaneous,  real-time  auditory  and  visual  infonnation  and  executing  specific 
and  complex  tasks  within  a  limited  and  often  immediate  time  frame.  As  can  be  surmised,  flying 
the  MQ-1  Predator  requires  a  high  level  of  skill  and  ability  as  a  pilot.3 


Nagy  JE,  Kalita  SW,  Eaton  G.  U.S.  Air  Force  unmanned  aircraft  systems  performance  analyses,  Predator  pilot 
front  end  analysis  (FEA)  report.  Brooks  City -Base  (TX):  311th  Human  Systems  Wing,  Performance  Enhancement 
Directorate;  2006.  Technical  Report  SURVIAC-TR-06-203.  Available  through  the  Defense  Technical  Information 
Center  to  U.S.  Government  agencies  and  their  contractors  only. 
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Comprehensive  qualitative  reviews  of  the  knowledge,  skills,  and  abilities  of  MQ-1 
Predator  pilots  have  identified  several  cognitive  aptitudes  as  key  to  performance,  including 
sustained  attention  and  concentration,  perceptual  reasoning  and  processing,  visual  memory  and 
learning,  spatial  and  symbolic  reasoning,  general  speed  and  accuracy  of  infonnation  processing, 
psychomotor  dexterity,  reaction  time,  as  well  as  visual  tracking,  searching,  and  scanning  [4].b 
Bailey  speculated  that  cognitive  aptitudes  (e.g.,  visual  perceptual  reasoning)  contributed  to  about 
two-thirds  of  the  factors  associated  with  MQ-1  Predator  pilot  performance.*3  The  results  are 
similar  to  other  qualitative  studies  postulating  on  the  cognitive  aptitudes  critical  to  the 
perfonnance  of  military  RPA  pilots  flying  large-sized  aircraft  at  medium  to  high  altitudes  across 
U.S.  and  foreign  airspace  [5].a,c  However,  the  research  studies  cited  above  regarding  cognitive 
predictors  of  perfonnance  are  based  on  theoretical  speculation  and  qualitative  reviews. 

Despite  the  limited  number  of  studies  with  objective  testing,  research  has  found  those 
who  enter  USAF  Predator  pilot  training  have  high  levels  of  pre-training  cognitive  abilities 
(intellectual  and  neuropsychological).  In  particular,  two  studies  revealed  MQ-1  Predator  RPA 
pilot  training  candidates  to  have  high  average  to  superior  levels  of  general  intellectual 
functioning,  speed  and  accuracy  of  infonnation  processing,  visual  reasoning  and  memory, 
psychomotor  reaction  time,  as  well  as  attention  and  concentration  when  compared  with  peers  in 
the  general  nonnative  population  [4,6].  These  findings  are  similar  to  the  cognitive  abilities  of 
USAF  pilots  who  fly  manned  airframes  [7-9]. 

However,  existing  literature  on  the  general  intelligence  and  cognitive  aptitudes  specific  to 
(and  predictive  of)  performance  outcomes  of  USAF  Predator  pilots  remains  limited  and 
controversial.  Anecdotal  discussions  with  RPA  Predator  line  and  training  leadership  reveal  a 
wide  range  of  divergent  opinions  regarding  cognitive  aptitudes  that  distinguish  those  with  the 
highest  levels  of  overall  training  perfonnance.  For  instance,  some  line  commanders  have 
postulated  that  faster  psychomotor  reaction  time  and  information  processing  speed,  as  well  as 
high  levels  of  memory,  distinguish  such  performers,  whereas  others  have  postulated  higher  levels 
of  attention,  spatial  analytical  abilities,  and  general  cognitive  functioning  are  the  key  aptitudes 
that  delineate  those  with  the  highest  overall  performance. 

However,  identifying  the  cognitive  aptitudes  that  delineate  (and  are  predictive  of  )  those 
with  overall  elite  training  performance  may  be  particularly  challenging  given  the  entire  pool  of 
training  candidates  has  high  average  to  superiors  levels  of  perfonnance  across  a  range  of 
cognitive  aptitudes  measured  via  tests  of  intelligence  and  neuropsychological  functioning  [4,6]. 
Furthennore,  RPA  candidates  are  selected  based  on  their  perfonnance  on  measures  of  general 
mental  ability  (i.e.,  Armed  Forces  Officer  Qualifying  Test,  Test  of  Basic  Attributes)  and 
academic  achievement.  As  a  result,  it  is  likely  that  any  study  attempting  to  identify  pre-training 
cognitive  aptitudes  that  delineate  those  with  the  highest  levels  of  overall  training  performance 
will  be  subjected  to  range  restriction. 


b  Bailey  M.  Predator  pilot  and  sensor  operator  selection  test  batteries.  United  Kingdom:  Cranwell  Royal  Air  Force 
Base;  2009.  Royal  Air  Force  Technical  Report.  Available  by  request  only. 

c  Nagy  JE,  Muse  K,  Eaton  G,  Phillips  A.  U.S.  Air  Force  unmanned  aircraft  systems  performance  analyses:  Global 
Hawk  pilot  and  sensor  operator  front  end  analysis  (FEA)  report.  Brooks  City-Base  (TX):  31 1th  Human  Systems 
Wing,  Performance  Enhancement  Directorate;  2007.  Technical  Report  SURVIAC-TR- 10-041.  Available  through 
the  Defense  Technical  Information  Center  to  U.S.  Government  agencies  and  their  contractors  only. 
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Nonetheless,  identifying  the  pre-training  aptitudes  that  distinguish  and  predict  those  with 
elite  levels  of  overall  training  performance  may  provide  new  insights  for  improving  selection  and 
aeromedical  screening  processes,  as  well  as  training  outcomes.  This  is  especially  important 
given  the  rising  dependency  upon  RPA  weapons  bearing  aircraft  for  supporting  multiple  and 
continuous  surveillance,  reconnaissance,  and  combat  weapons-strike  operations  throughout  the 
globe  and  the  importance  of  recruiting  and  filling  the  increasing  number  of  MQ- 1  Predator  RPA 
pilot  vacancies  affecting  readiness  levels  of  units  across  the  USAF  [2,3]. 

The  purpose  of  this  exploratory  study  is  to  (a)  assess  differences  among  candidates 
grouped  according  to  elite  versus  adequate  training  performance  on  measures  of  intelligence  and 
neuropsychological  testing  and  (b)  identify  cognitive  aptitudes  predictive  of  perfonnance 
outcomes.  It  is  hypothesized  those  with  elite  perfonnance  will  have  higher  levels  of  cognitive 
functioning  in  the  areas  of  visual-perceptual  functioning  (i.e.,  visual  reasoning,  visual  memory, 
and  learning)  and  speed  of  infonnation  processing  due  to  the  visually  complex  and  demanding 
nature  of  flying  the  MQ-1  Predator  and  the  results  of  comprehensive  qualitative  reviews  [4,5]b 
emphasizing  the  importance  of  such  abilities. 

3.0  METHODS 

3.1  Participants 

In  total,  89  participants  were  selected  from  a  pool  of  235  USAF  MQ-1  Predator  pilot 
training  candidates  who  completed  Initial  Qualification  Training  (IQT).  Participants  were 
selected  based  upon  their  average  daily  scores  throughout  training.  High  perfonners  were 
identified  as  participants  with  scores  in  the  upper  quintile  (i.e.,  top  20%)  of  those  who  completed 
training.  Adequate  performers  were  identified  as  participants  with  scores  in  the  lowest  quintile 
(bottom  20%)  of  those  who  completed  training. 

In  total,  46  out  of  235  (19.57%)  pilots  who  completed  IQT  were  identified  as  the  highest 
(upper  quintile)  in  overall  training  perfonnance.  This  group  consisted  of  44  males  (95.65%)  and 
2  females  (4.35%).  A  total  of  42  high  perfonners  (91.30%)  reported  Caucasian  as  their 
ethnicity/race,  and  1  high  performer  (2.17%)  reported  Asian  as  his  ethnicity/race.  The  average 
age  for  this  group  was  2 1 .98  years  (standard  deviation  ( SD )  =  2.62),  with  a  range  of  19  -  32 
years.  Data  were  not  available  for  ethnicity/race  on  three  elite  performers  (6.52%)  and  for  age 
on  two  high  performers  (4.35%). 

Overall,  43  out  of  235  (18.30%)  pilots  who  completed  IQT  were  identified  with  adequate 
perfonnance  (lower  quintile).  This  group  consisted  of  40  males  (93.02%)  and  3  females 
(6.98%).  A  total  of  34  adequate  perfonners  (79.07%)  reported  Caucasian  as  their  ethnicity/race, 
2  (4.65%)  reported  Asian,  2  (4.65%)  reported  African,  and  1  (2.32%)  reported  Hispanic.  The 
average  age  for  this  group  was  23.86  years  (SD  =  2.98),  with  a  range  of  19  -  32  years.  Data 
were  not  available  for  ethnicity/race  on  four  adequate  perfonners  (9.30%)  and  for  age  on  seven 
adequate  performers  (16.28%). 

3.2  Measures 

3.2.1  Intelligence  Testing.  The  Multidimensional  Aptitude  Battery  II  (MAB-II)  assesses 
general  intelligence  with  five  distinct  verbal  intelligence  subscales,  five  distinct  performance 
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intelligence  subscales,  a  verbal  intelligence  quotient,  a  perfonnance  intelligence  quotient,  and  an 
overall  general  intelligence  quotient  [10]. 

The  MAB-II  is  a  computer-based  test  for  administration  and  scoring  and  is  a  part  of 
baseline  testing  during  medical  flight  screening  for  all  USAF  RPA  pilots  [11].  The  measure  is 
based  upon  the  structure  and  content  of  the  Wechsler  Adult  Intelligence  Scale  and  Wechsler’s 
theory  of  intelligence  [12].  The  MAB-II  manual  has  well-documented  internal  consistency 
reliability  of  0.94  -  0.97  on  the  subscales  and  0.92  -  0.95  on  the  intelligence  quotients  and  test- 
retest  reliability  ranging  from  0.83  -  0.97  on  the  subscales  and  0.94  -  0.97  on  the  intelligence 
quotients  [10].  For  the  general  population  normative  scores,  the  standardized  scores  for  the 
MAB-II  subscales  have  a  mean  of  50  (SD  =  10),  the  intelligence  quotients  have  a  mean  of  100 
(SD  =  15),  and  they  have  been  statistically  corrected  for  age  [10]. 

3.2.2  Neuropsychological  Screening.  The  MicroCog  was  used  to  assess  neuropsychological 
functioning  [13].  Eighteen  sub  tests  are  combined  to  yield  five  first-level  indices  representative 
of  five  neuropsychological  domains:  Attention/Mental  Control,  Memory, 

Reasoning/Calculation,  Spatial  Processing,  and  Reaction  Time.  Two  second-level  indices  of 
Information  Processing  Speed  and  Infonnation  Processing  Accuracy  are  also  derived  from  the 
subtests  but  take  into  account  overall  speed  and  accuracy  rather  than  specific  domains.  Two 
third-level  indices  of  General  Cognitive  Functioning  and  General  Cognitive  Proficiency  are  also 
assessed.  General  Cognitive  Functioning  is  derived  from  the  two  second-level  indices  and 
General  Cognitive  Proficiency  is  derived  from  the  individual  subtest  proficiency  scores, 
including  both  infonnation  processing  speed  and  accuracy  with  weighting  on  accuracy  of 
processing  [13]. 

The  MicroCog  is  a  computer-based  test  for  administration  and  scoring  and  is  also  a  part 
of  medical  baseline  testing  during  medical  flight  screening  for  all  USAF  RPA  pilots  [6,1 1]. 
Multiple-choice  items  with  varying  numbers  of  response  options  and  free-response  items  are 
presented.  MicroCog  normative  scores  for  the  five  first-level  indices,  two  second-level  indices, 
and  two  third-level  indices  each  have  a  mean  of  100  (SD  =15)  and  have  been  statistically 
adjusted  for  age  and  level  of  education  (bachelor’s  degree  in  the  current  study).  The  manual  for 
the  MicroCog  documents  internal  consistency  reliability  ranging  from  0.83  -  0.95  and  test-retest 
reliability  ranging  from  0.73  -  0.99  on  the  indices  for  the  standard  form  [13,14]. 

3.2.3  Training  Outcomes.  Training  outcomes  from  the  IQT  course  for  the  MQ-1  Predator 
aircraft  were  used  in  this  study.  At  the  time  of  this  study,  the  IQT  course  consisted  of  45  training 
days,  with  21  ground  training  days  and  24  flying  training  days,  for  a  total  of  174.5  training  hours. 
RPA  pilot  trainees  were  rated  throughout  each  simulator  and  live  flight  sortie  (20  sorties  total) 
using  individual  mission  grade  sheets  that  assess  several  areas  of  functioning:  mission 
planning/preparation,  administrative  checklist  procedures,  display  interpretation,  display/data 
manipulation,  pedestal  functions,  system  operations,  aircraft  control,  navigation  procedures, 
communication  and  coordination  procedures,  general  knowledge,  safety,  discipline,  emergency 
mission  procedures  and  knowledge,  etc.d  The  RPA  pilot  trainee’s  performance  was  evaluated  in 
a  checklist  format  on  the  mission  grade  sheet  across  multiple  areas  of  functioning  and  an  overall 
perfonnance  score  for  that  sortie  was  rated  on  a  scale  of  0  to  4: 


d  Air  Combat  Command.  MQ-1B  qualification  training  [Unpublished  technical  report].  Langley  AFB  (VA): 
Department  of  the  Air  Force,  Headquarters  Air  Combat  Command;  2008. 
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•  0  (lack  of  ability  or  knowledge;  dangerous/unsafe  performance) 

•  1  (perfonnance  is  safe,  but  proficiency  is  limited;  makes  errors  of  omission  or  commission) 

•  2  (performance  is  essentially  correct;  recognizes  and  corrects  errors) 

•  3  (performance  is  correct,  efficient,  skillful,  and  without  hesitation) 

•  4  (performance  reflects  an  unusually  high  degree  of  ability) 

The  training  candidates  must  achieve  a  proficiency  score  of  2  or  greater  on  each  of  the 
elements  to  successfully  complete  the  course.  Training  outcomes  for  the  MQ-1  IQT  course 
consisted  of  averaged  scores  from  individual  mission  grade  sheets  from  20  simulator  and  live 
flight  sorties.  Each  sortie  was  graded  by  one  instructor  and  each  candidate  was  graded  by 
multiple  instructors  throughout  his  or  her  training. 

3.3  Procedure 

The  MQ-1  Predator  pilots  in  this  study  were  administered  the  MAB-II  and  MicroCog  as  a 
routine  part  of  medical  flight  screening  prior  to  attending  training.  Test  administration  occurred 
at  two  locations  (USAF  School  of  Aerospace  Medicine  and  the  USAF  Academy).  All  candidates 
were  tested  under  a  standardized  set  of  instructions  and  conditions.  Training  candidates  were 
infonned  of  the  potential  uses  of  their  baseline  testing  for  medical  and  research  purposes.  Each 
pilot’s  test  scores  were  archived  and  combined  later  with  training  numerical  rating  outcome 
grades  and  a  “rack  and  stack”  list  assessing  how  each  pilot  performed  relative  to  all  other  pilots 
after  completion  of  IQT. 

3.4  Data  Analysis 

The  analyses  in  this  study  were  generated  using  SAS®  software,  version  9.3  (SAS 
Institute,  Cary,  NC).  The  dataset  consisted  of  235  MQ-1  Predator  pilot  training  candidates  who 
passed  IQT  training.  Participants  were  sorted  in  descending  order  on  their  average  MQ- 1  IQT 
daily  score  and  dichotomized  into  two  groups  based  on  a  cut-off  representing  a  change  in  score 
closest  to  the  lowest  and  highest  quintiles. 

Descriptive  statistics  were  computed  for  the  high  and  adequate  perfonnance  groups. 
Correlations  were  run  on  the  MAB-II  and  MicroCog  variables  with  the  outcome  variable  (high 
vs.  adequate  perfonnance). 

To  determine  which  aptitudes  were  predictive  of  high  perfonners,  stepwise  binary  logistic 
regressions  were  used  to  retain  predictor  effects  titrating  from p  <  .05  to  p  <  .30,  by  incremental 
units  of  .05  [15].  While  hierarchical  logistic  regression  utilizing  all  predictors  is  prefened  in  the 
literature,  the  current  study  was  limited  by  the  small  sample  sizes  in  each  group.  While  the 
standard  requirement  is  10  records  per  outcome  variable  for  logistic  regressions  [15],  research 
has  shown  that  less  records  per  outcome  are  possible  if  the  outcome  groups  are  equal  [16].  It  has 
been  recommended  that  as  low  as  five  records  per  outcome  variable  are  adequate  when 
interpreted  with  caution  [17].  With  the  smallest  group  (n  =  43),  five  records  per  predictor 
allowed  for  8.6  predictors.  Parameters  to p  <  .50  have  been  found  in  the  literature  in  small 
sample  sizes  [15],  but  the  model  with p  <  .30  was  used  to  account  for  the  maximum  number  of 
predictors  without  over  fitting  the  model. 
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Logistic  regressions  were  run  individually  using  the  10  MAB-II  subscales,  then  for  the  5 
MicroCog  first-level  and  2  second-level  indices,  and  finally  for  the  MAB-II  and  MicroCog 
combined.  The  higher  level  composite  scores  were  not  used  in  the  predictive  models  because 
they  are  formulated  from  the  lower  level  measures  (i.e.,  MAB-II  intelligence  quotients  and 
MicroCog  third-level  indices).  Selection  of  variables  to  remain  in  the  model  was  based  upon  the 
relative  impact  of  each  variable  to  the  increased  incremental  validity  of  the  model.  This  was 
assessed  by  reviewing  the  R~  at  each  step  of  the  model,  sorting  the  absolute  value  of  the 
standardized  beta  estimates  and  the  results  of  the  receiver  operating  curve  estimates.  Goodness- 
of-fit  statistics  were  perfonned  for  each  model.  Negative  and  positive  predictive  value  rates 
were  calculated  from  the  logistic  regression  leave-one-out  cross-validation  classification  tables. 

Multivariate  range  restriction  was  also  considered.  With  a  sample  consisting  of  only  pilot 
training  candidates  who  passed  training  (and  not  including  those  who  failed  training),  indirect 
range  restriction  regarding  perfonnance  on  measures  of  cognitive  aptitude  is  assumed  to  exist 
due  to  selection  requirements  [18].  The  effects  of  direct  range  restriction  were  also  noted, 
because  this  study  truncated  the  dataset  from  the  original  ( n  =  235)  to  only  the  top  and  bottom 
quintiles.  A  comparison  of  the  predictor  intercorrelations  and  predictor  correlations  with  the 
outcome  variable  for  the  original  dataset  (splitting  the  dataset  at  the  50th  percentile)  and  the 
truncated  dataset  (top  and  bottom  quintiles)  was  performed.  When  ranking  the  correlations  of 
the  predictors  to  the  outcome  variable  in  descending  order,  the  order  of  predictors  did  not  change 
when  comparing  the  results  of  the  original  dataset  and  the  truncated  dataset.  This  indicates  that 
while  corrections  for  direct  range  restriction  may  increase  the  correlations  with  the  outcome 
variable,  the  same  predictors  would  have  emerged  from  a  corrected  model  because  the  predictors 
with  the  top  relative  contribution  to  the  model  would  have  been  the  same.  Additionally,  effects 
of  range  enhancement,  or  choosing  only  groups  from  the  top  and  bottom  quintiles,  may  have 
resulted  in  an  increase  in  correlations  with  the  outcome  variable  when  compared  to  correlations 
in  the  nonnative  population.  Guidelines  for  the  minimum  n  required  for  indirect  range 
restriction  were  used,  based  on  required  power,  restricted  predictor  validity,  predictor 
intercorrelations,  unrestricted  predictor  validity,  selection  ratio,  and  criterion  reliability  [19]. 
Accounting  for  the  selection  ratio  in  the  truncated  dataset  (selection  ratio  of  .40  to  account  for 
the  two  sets  of  quintiles)  and  the  criterion  reliability  of  0.63,  the  minimum  required  sample  size 
was  129.  Additionally,  this  required  sample  size  may  still  be  underestimated,  based  on 
corrections  to  the  formula  by  Raju,  Edwards,  and  LoVerde  [20].  With  the  final  sample  size  in 
the  current  study  of  89  not  meeting  the  minimum  required  sample  size,  logistic  regressions 
utilizing  multivariate  range  restriction  corrections  are  not  reported.  Multivariate  range  restriction 
correlation  adjustments  were  calculated  [21],  and  subsequent  logistic  regression  analyses  were 
conducted  on  uncorrected  correlations.  Corrections  for  dichotomizing  the  criteria  were  also 
considered,  but  are  not  reported  because  of  the  small  sample  size  [22]. 

4.0  RESULTS 

It  is  important  to  note  all  participants  in  this  study  completed  MQ-1  Predator  pilot 
training  before  being  assigned  to  their  operational  unit  for  combat  mission  readiness  training. 

All  pilots  were  deemed  proficient  and  met  qualifications  to  graduate  training.  The  group  average 
training  score  for  the  elite  performers,  taken  as  an  average  of  daily  individual  grade  sheets,  was 
2.19  ( SD  =  0.10),  with  a  range  of  2.07  -  2.53.  The  group  average  training  score  for  the  adequate 
performers  was  1.76  (SD  =  0.1 1),  with  a  range  of  1.50  -  1.86.  While  adequate  perfonnance 
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represented  safe,  effective  flying  with  limited  proficiency,  elite  performance  represented  safe, 
effective  flying  along  with  proficient  corrections  of  errors  and  high  levels  of  skill. 

Tables  1  and  2  provide  descriptive  statistics  on  the  MAB-II  and  MicroCog  and 
correlations  with  the  outcome  variable  (high  vs.  adequate  performance).  Both  adequate  and  elite 
performers  scored  at  least  one  standard  deviation  higher  than  the  age-corrected  normative 
population  (mean  ( M)  =  100,  SD=  15)  on  the  three  intelligence  quotients  measured  by  the 
MAB-II.  In  addition,  elite  performers  scored  at  least  one  standard  deviation  higher  than  the 
normative  population  (M  =  50,  SD  =  10)  on  all  five  verbal  subscales  and  on  four  performance 
subscales.  Adequate  perfonners  also  scored  at  least  one  standard  deviation  higher  than  the 
normative  population  on  one  verbal  subscale  and  three  performance  subscales.  Standard 
deviations  for  both  groups  on  the  intelligence  quotients  and  subscales  were  smaller  than  the 
normative  population,  showing  less  variance  in  scores  for  each  group.  Correlations  with  elite 
versus  adequate  performance  were  significant  aip  <  .01  for  Verbal  IQ;  the  Information, 
Similarities,  and  Vocabulary  verbal  subscales;  and  the  Digit  Symbol  performance  subscale.  A 
correlation  was  significant  at  p  <  .05  for  the  Arithmetic  verbal  subscale. 

Table  1.  MAB-II  Descriptive  Statistics  and  Correlations  with  Outcome 
(High  vs.  Adequate  Performance) 


Quotients  and 
Subscales 

High 

Performers 

M  (SD) 

(n  =  46) 

Adequate 

Performers 

M  (SD  ) 

(n  =  43) 

r 

rc 

Intelligence  Quotients 

Full  Scale 

120.26  (16.83) 

118.30  (6.60) 

.08 

Verbal 

121.65 

(5.34) 

116.33  (5.55) 

.44a 

Perfonnance 

120.17 

(7.39) 

117.88  (9.47) 

.14 

Verbal  Subscales 

Information 

69.13 

(4.36) 

64.81  (5.70) 

.40a 

.66 

Comprehension 

60.33 

(3.88) 

58.77  (4.85) 

.18 

.60 

Arithmetic 

61.07 

(7.26) 

57.95  (7.18) 

.21b 

.48 

Similarities 

61.48 

(3.80) 

58.81  (4.51) 

.31a 

.57 

Vocabulary 

62.17 

(6.49) 

57.30  (6.34) 

.36a 

.60 

Performance  Subscales 

Digit  Symbol 

68.04 

(6.07) 

63.21  (9.12) 

.30a 

.54 

Picture  Completion 

60.91 

(6.32) 

60.44  (6.46) 

.04 

.19 

Spatial  Analyses 

60.72 

(6.96) 

59.12  (7.03) 

.12 

.36 

Picture  Arrangement 

50.72 

(7.35) 

50.53  (8.04) 

.01 

.21 

Object  Assembly 

62.24 

(5.13) 

61.65  (5.40) 

.06 

.32 

Note:  Column  headings  indicate  observed  correlations  (r)  and  correlations  corrected  for  range 
restriction  (rc). 

><.01. 

bp  <  .05.  95%  Cl  =  95%  confidence  interval.  Scores  are  corrected  for  age. 
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Both  adequate  and  elite  performers  had  mean  group  scores  within  one  standard  deviation 
of  the  age-  and  education-corrected  normative  population  (M  =  100,  SD  =  15)  [13]  on  each  of  the 
neuropsychological  indices  measured  by  the  MicroCog.  Correlations  with  high  vs.  adequate 
performance  were  significant  at/?  <  .01  for  the  Memory  first-level  index,  Information  Processing 
Speed  second-level  index,  and  General  Cognitive  Proficiency  third-level  index.  Correlations 
significant  at  p  <  .05  were  the  Attention/Mental  Control  and  Reasoning/Calculation  first-level 
indices  and  the  General  Cognitive  Functioning  third-level  index. 

Stepwise  logistic  regressions  were  run  using  the  10  MAB-II  subscales,  the  5  MicroCog 
first-level  and  2  second-level  indices,  and  then  the  MAB-II  and  MicroCog  measures  combined. 
Table  3  shows  the  final  predictors  for  each  model,  Wald  coefficients,  and  standard  estimates. 

The  leave-one-out  cross-validation  classification  table  for  the  MAB-II  subscales  stepwise  logistic 
regression  revealed  71.74%  of  elite  performers  and  55.81%  of  adequate  performers  were 
correctly  classified.  The  negative  predictive  value  was  64.86%  and  the  positive  predictive  value 
was  63.46%.  The  model  had  acceptable  goodness-of-fit  when  assessing  area  under  the  curve 
(AUC)  =  0.78.  Hosmer  and  Lemeshow  x2(8)  =  4.06,/?  =  .85  demonstrated  acceptable  goodness- 
of-fit.  The  stepwise  logistic  regression  for  the  MicroCog  first-  and  second-level  indices  revealed 
76.09%  of  elite  performers  and  53.49%  of  adequate  performers  were  correctly  classified.  The 
negative  predictive  value  was  67.65%  and  the  positive  predictive  value  was  63.64%.  The  model 
had  acceptable  goodness-of-fit  when  assessing  AUC  =  0.76  and  Hosmer  and  Lemeshow  y2(8)  = 
6.02,/?  =  .65. 

Table  2.  MicroCog  Descriptive  Statistics  and  Correlations  with  Outcome 
(High  vs.  Adequate  Performance) 


Indices 

High 

Performers 
Mean  (SD) 

(n  =  46) 

Adequate 
Performers 
Mean  (SD) 

(n  =  43) 

r 

rc 

First  Level 

Attention/Mental  Control 

105.52  (11.62) 

99.12(11.44) 

IT 

.30 

Reasoning/  C  alculation 

99.04(11.83) 

94.19(11.03) 

IT 

.26 

Memory 

113.93  (15.56) 

104.51  (14.23) 

.30b 

.29 

Spatial  Processing 

107.61  (8.53) 

104.02(12.88) 

.17 

.25 

Reaction  Time 

99.00(13.61) 

99.00(13.43) 

.00 

.00 

Second  Level  (Information) 

Processing  Speed 

109.20(10.26) 

97.42(14.17) 

.44b 

.44 

Processing  Accuracy 

98.67(14.67) 

98.77(12.34) 

-.00 

-.07 

Third  Level  (General) 

Cognitive  Functioning 
Cognitive  Proficiency 

112.09(14.03) 
106.91  (9.83) 

105.95  (14.45) 
100.44  (9.62) 

.21a 

.32b 

Note:  Column  headings  indicate  observed  correlations  (r)  and  correlations  corrected  for  range  restriction 
(rc).  Scores  are  corrected  for  age  and  education. 

7?  <  .05. 
b/?<.01. 
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Table  3.  High  vs.  Adequate  Overall  MQ-1  Predator  Training  Performance  Logistic 

Regression  Model 


Step 

R2 

Predictors 

Wald 

P 

Std  Estimate 

1 

.324 

MAB-II 

Information 

13.99 

.00 

0.32 

Vocabulary 

2.55 

.11 

0.32 

Digit  Symbol 

3.49 

.06 

0.30 

Arithmetic 

1.87 

.17 

0.26 

Spatial  Analyses 

1.43 

.23 

0.19 

2 

.291 

MicroCog 

Information  Processing  Speed 

16.88 

.00 

0.55 

Memory 

2.99 

.08 

0.24 

3 

.548 

MAB-II  &  MicroCog 

Information  Processing  Speed 

16.88 

.00 

1.07 

Vocabulary 

13.64 

.00 

0.52 

Spatial  Analyses 

3.53 

.06 

0.40 

Arithmetic 

2.48 

.12 

0.40 

Digit  Symbol 

2.37 

.12 

0.37 

Reasoning/Calculation 

2.22 

.14 

0.34 

Information 

2.91 

.09 

0.33 

When  the  MicroCog  was  added  to  the  MAB-II,  an  additional  25.72%  of  variance  was 
explained.  With  a  criterion  cut-off  score  of  .5,  the  stepwise  logistic  regression  for  the  combined 
model  revealed  71.74%  of  elite  perfonners  (33  out  of  46)  and  62.79%  of  adequate  performers 
(27  out  of  43)  were  correctly  classified.  Positive  predictive  value  is  an  assessment  of  how  well 
the  model  successfully  predicted  true  positives  from  all  those  predicted  to  be  true  positives.  If 
the  16  false  positives  (adequate  perfonners  who  were  predicted  to  be  elite  performers)  are  added 
to  the  33  correctly  classified  elite  perfonners,  there  are  49  predicted  elite  performers.  The 
positive  predictive  value  for  elite  performers  equals  67.35%,  or  the  number  of  elite  perfonners 
conectly  classified  by  the  total  predicted  to  be  elite  perfonners.  The  negative  predictive  value 
was  67.50%.  While  the  negative  predictive  value  was  similar  to  the  MicroCog  only  model,  the 
positive  predictive  value  for  the  combined  model  increased  when  compared  to  both  the  MAB-II 
only  and  MicroCog  only  models.  While  the  Hosmer  and  Lemeshow  goodness-of-fit  statistic, 
X2(8)  =  9.67,  p  =  .29,  was  low,  stability  of  the  predictive  model  was  shown  by  acceptable  AUC  = 
0.88  and  a  less  than  10%  shrinkage  in  overall  classification  ability  from  the  initial  to  the  cross- 
validated  leave-one-out  classification. 

5.0  DISCUSSION 

It  is  important  to  note  all  MQ-1  Predator  pilots  in  this  study  completed  MQ-1  Predator 
pilot  training  (which  occurs  before  being  assigned  to  their  operational  unit  for  combat  mission 
readiness  training,)  were  deemed  proficient  in  the  pilot  skills,  and  met  qualifications  to  graduate 
training.  While  adequate  performance  represented  safe,  effective  flying  with  limited  proficiency, 
elite  performance  represented  safe,  effective  flying  along  with  proficient  corrections  of  errors 
and  high  levels  of  skill. 
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The  results  revealed  both  elite  and  adequate  performers  scored  higher  than  the  general 
civilian  normative  population  on  intelligence  testing  with  less  variance.  This  resulted  in  an 
attenuated  distribution  (e.g.,  range  restriction)  of  intelligence  scores  among  Predator  pilots  from 
both  groups  when  compared  with  the  nonnative  population.  However,  the  scores  from  both 
groups  were  similar  to  USAF  pilots  of  manned  airframes  [7,9]  and  to  a  previous  study  on  MQ-1 
Predator  RPA  pilot  training  candidates  [6].  Additionally,  those  with  elite  training  perfonnance 
scored  higher  on  all  five  of  the  verbal  ability  subscales  (e.g.,  Infonnation,  Comprehension, 
Arithmetic,  Similarities,  and  Vocabulary)  and  on  only  one  out  of  live  perfonnance-based 
subscales  (i.e.,  Digit  Symbol)  when  compared  to  those  with  adequate  performance  scores.  As  a 
group,  those  with  elite  training  performance  had  higher  levels  of  intelligence  in  the  areas  of 
general  knowledge,  social  reasoning  and  comprehension,  numerical  processing  and  reasoning, 
general  conceptual  and  verbal  reasoning,  as  well  as  visual  learning  and  processing  speed. 

In  regard  to  neuropsychological  testing,  there  also  was  less  variance  in  the  scores  between 
both  groups  of  pilots  when  compared  with  peers  in  the  general  civilian  normative  population. 

The  scores  from  both  groups  were  similar  to  USAF  manned  airframe  pilots  [7,9].  However,  as  a 
group,  pilots  with  elite  training  performance  also  scored  higher  on  neuropsychological  measures 
of  attention  and  concentration,  reasoning,  memory,  spatial  processing,  as  well  as  speed  of 
infonnation  processing.  These  areas  of  neuropsychological  functioning  were  postulated  as 
critical  to  adaptation  to  Predator  pilot  job  skills  and  demands  in  an  earlier  qualitative  study 
involving  interviews  with  subject  matter  experts  [4]. 

5,1  Intelligence  and  Neuropsychological  Predictors  of  High  vs.  Adequate  Performance 

The  findings  of  Infonnation  Processing  Speed  and  Vocabulary  (a  measure  of  verbal 
ability)  as  significant  predictors  of  performance  suggest  those  within  the  elite  performance  group 
had  a  higher  capacity  to  effectively  process  and  respond  to  multiple  inputs  of  visual  and  auditory 
infonnation  with  a  more  robust  set  of  vocabulary  skills  to  draw  from  for  effectively  articulating 
oneself  and  communicating  with  others  in  regard  to  information,  tasks,  and  procedures.  Such 
aptitudes  are  reasonably  perceived  as  essential  for  minimizing  the  risk  for  mishaps  and 
responding  to  both  routine  and  emergent  (and  often  unpredictable)  conditions  during  flight. 

The  findings  of  Spatial  Analyses,  Arithmetic,  and  Digit  Symbol  measures  as  significant 
predictors  of  perfonnance  suggest  those  in  the  elite  performance  group  had  higher  visual-spatial 
reasoning,  numerical  processing,  and  working  memory  aptitudes  and  are  consistent  with 
previous  research  identifying  such  aptitudes  as  an  area  of  functioning  distinguishing  USAF  pilot 
candidates  [11].  Such  aptitudes  are  reasonably  perceived  to  influence  how  quickly  and 
accurately  a  training  candidate  can  calculate  the  position  and  distance  of  the  aircraft  between 
fixed  and  moving  targets,  as  well  as  calculating,  understanding,  and  remembering  spatial 
relations  in  general.  These  aptitudes  also  appear  particularly  important  for  effectively 
responding  to  the  visually  complex  and  highly  demanding  spatial  and  numerical  calculation  tasks 
during  flight,  as  well  as  the  construction  of  mental  representations  of  object  configurations  from 
images  on  several  screens  representing  constantly  changing  geographical  images  and  numerical 
data. 

The  finding  of  the  neuropsychological  Reasoning  and  Calculation  index  as  predictive  of 
perfonnance  suggests  those  with  elite  performance  had  higher  capacity  for  concept  fonnation, 
cognitive  flexibility,  inductive  reasoning,  and  perfonning  mental  calculations.  Such  aptitudes 
also  represent  a  higher  capacity  for  critical  thinking  and  problem  solving  within  a  condensed, 
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limited  period  of  time.  Processing  and  identifying  infonnation  in  a  logical  fashion  and 
effectively  labelling,  prioritizing,  and  attending  to  important  information  when  inundated  with 
large  amounts  of  visual  and  auditory  data  are  key  to  perfonnance.  This  is  particularly  important 
to  analyzing  various  facts  and  points  of  view  and  quickly  identifying  errors  or  areas  of  concern 
related  to  specific  flying  tasks. 

The  finding  of  the  Information  subscale  (general  fund  of  knowledge  for  world-related 
facts  and  infonnation)  as  a  significant  predictor  of  performance  was  unexpected  given  that 
general  fund  of  knowledge  is  often  influenced  by  educational  level  and  all  USAF  pilots  are 
college  graduates.  However,  further  analyses  of  this  standing  suggest  those  with  elite 
perfonnance  have  a  higher  capacity  for  storing  and  retrieving  infonnation  in  general.  This  may 
be  particularly  important  when  learning  new  tasks  and  procedures  and  fanning  an  understanding 
of  conditions  as  they  emerge. 

Although  intelligence  testing  alone  explained  29.1 1%  of  the  variance  in  perfonnance,  the 
addition  of  neuropsychological  testing  increased  the  variance  to  54.83%.  Of  the  elite  perfonners, 
the  model  conectly  classified  7 1 .74%  as  elite  performers.  The  results  suggest  that  a  collective 
combination  of  intelligence  and  neuropsychological  testing  tapping  into  speed  of  infonnation 
processing,  visual-spatial  analyses,  reasoning,  numerical  processing,  visual  memory,  and 
vocabulary  may  help  to  improve  aeromedical  screening  and  personnel  classification  processes 
for  identifying  recruits  with  the  “right  stuff’  for  MQ-1  Predator  RPA  pilot  training. 

5.2  Limitations 

This  study  had  limitations  that  warrant  discussion.  First,  the  group  sample  sizes  for  both 
the  adequate  and  high  perfonnance  groups  were  small  ( less  than  50).  This  may  have  hindered 
the  ability  to  perform  multivariate  range  restriction  corrections.  Performing  range  restriction 
adjustments  on  small  sample  sizes  may  lead  to  results  with  a  50%  chance  or  less  of  detecting 
validity,  if  validity  exists.  Second,  caution  should  be  given  to  generalizing  the  results  from  this 
study  to  other  RPA  pilot  career  fields  within  civilian  and  military  industries.  Performance  and 
task  requirements  for  other  RPA  airframes  (such  as  low-altitude,  non-weapons-bearing 
platfonns)  may  differ  significantly.  Third,  repeated  studies  are  needed  to  assess  for  differences 
regarding  minority  group  status.  Studies  evaluating  cognitive  perfonnance  should  remain 
sensitive  to  differences  between  minority  groups  to  ensure  objective  and  unbiased  decision 
making  when  utilizing  the  study  results  for  modifying  selection  and  aeromedical  screening 
processes.  Fourth,  the  inter-rater  reliability  between  instructors  on  the  average  daily  training 
scores  was  not  accessed  because  the  data  in  the  current  study  included  an  averaged  score  and  not 
daily  scores  or  information  on  the  grading  instructor.  Additional  studies  are  needed  to  replicate 
these  findings  and  provide  more  confidence  to  the  current  results. 
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5.3  Conclusions 


MQ-1  Predator  RPA  pilots  are  required  to  perform  complex  decision  making,  analyze 
real-time  auditory  and  visual  information,  and  understand  and  perform  effective  task 
prioritization  in  a  high-demand,  24-hour  operational  environment.  The  results  of  this  study 
reveal  cognitive  and  neuropsychological  aptitudes  have  a  key  role  in  performance  outcomes  for 
those  pursuing  this  unique  aviation-related  career  field.  The  results  suggest  that  including  the 
assessment  of  key  areas  of  psychological  functioning  identified  in  this  study  may  help  to 
improve  personnel  selection  as  well  as  aeromedical  screening  procedures. 

6.0  REFERENCES 

1 .  U.S.  Air  Force.  The  U.S.  Air  Force  remotely  piloted  aircraft  and  unmanned  aerial  vehicle 
strategic  vision.  Washington  (DC):  Department  of  the  Air  Force;  2005.  Research  Paper  1. 
[Accessed  Retrieved  from  1  Dec  2014].  Available  from  http://digitalcommons.unl.edu/cgi/ 
viewcontent.cgi?article=1000&context=usafresearch  . 

2.  Hoagland  BT.  Manning  the  next  unmanned  Air  Force:  developing  RPA  pilots  of  the  future. 
Washington  (DC):  Center  for  21st  Century  Security  and  Intelligence,  The  Brookings  Institute; 
2013.  [Accessed  1  Dec  2014].  Available  from  http://www.brookings.edu/research/papers/ 
2013/08/06-air-force-drone-pilot-development-hoagland. 

3.  Government  Accountability  Office.  Air  Force:  actions  needed  to  strengthen  management  of 
unmanned  aerial  system  pilots.  Washington  (DC):  GAO;  2014.  Report  GAO-14-316. 

4.  Chappelle  W,  McDonald  K,  McMillan  K.  Important  and  critical  psychological  attributes  of 
USAF  MQ-1  Predator  and  MQ-9  Reaper  pilots  according  to  subject  matter  experts.  Wright- 
Patterson  AFB  (OH):  U.S.  Air  Force  School  of  Aerospace  Medicine;  2011.  Technical  Report 
AFRL-SA-WP-TR-201 1-0002. 

5.  Paullin  C,  Ingerick  M,  Trippe  MD,  Wasko  L.  Identifying  best  bet  entry-level  selection 
measures  for  US  Air  Force  remotely  piloted  aircraft  (RPA)  pilot  and  sensor  operator  (SO) 
occupations.  Randolph  AFB  (TX):  Air  Force  Center  Personnel  Center;  2011.  Technical 
Report  AFCAPS-FR-201 1-0013.  [Accessed  1  Dec  2014].  Available  from  http://oai.dtic.miF 
oai/oai?verb=getRecord&metadataPrefix=html&identifier=ADA554209. 

6.  Chappelle  W,  Tran  NVT,  Thompson  W,  Goodman  T,  Hyde  K.  Intelligence  and 
neuropsychological  aptitude  testing  of  U.S.  Air  Force  MQ-1  Predator  pilot  training 
candidates.  Wright-Patterson  AFB  (OH):  U.S.  Air  Force  School  of  Aerospace  Medicine; 
2012.  Technical  Report  AFRL-SA-WP-TR-201 3-0003. 

7.  Chappelle  W,  Ree  MJ,  Barto  EL,  Teachout  MS,  Thompson  WT.  Joint  use  of  the  MAB-II  and 
MicroCog  for  improvements  in  the  clinical  and  neuropsychological  screening  and 
aeromedical  waiver  process  of  rated  USAF  pilots.  Brooks  City-Base  (TX):  U.S.  Air  Force 
School  of  Aerospace  Medicine;  2010.  Technical  Report  AFRL-SA-BR-TR-20 10-0002. 

8.  King  RE,  Barto  E,  Ree  MJ,  Teachout  MS,  Retzlaff  PD.  Compilation  of  pilot  cognitive  ability 
norms.  Wright-Patterson  AFB  (OH):  U.S.  Air  Force  School  of  Aerospace  Medicine;  2011. 
Technical  Report  AFRL-SA-WP-TR-201 2-0001. 

9.  King  RE,  Carretta  TR,  Retzlaff  P,  Barto  E,  Ree  MJ,  Teachout  MS.  Standard  cognitive 
psychological  tests  predict  military  pilot  training  outcomes.  Aviation  Psychology  and 
Applied  Human  Factors.  2013;  3(l):28-38. 


12 


DISTRIBUTION  STATEMENT  A.  Approved  for  public  release. 


Cleared,  88PA,  Case  #2015-1652. 


10.  Jackson  DN.  Multidimensional  Aptitude  Battery-II:  manual.  Port  Huron  (MI):  Sigma 
Assessment  Systems,  Inc.;  2003. 

11.  Chappelle  W,  Heerema  BD,  Thompson  WT.  Factor  analysis  of  computer-based 
Multidimensional  Aptitude  Battery-Second  Edition  intelligence  testing  from  rated  U.S.  Air 
Force  pilots.  Wright-Patterson  AFB  (OH):  U.S.  Air  Force  School  of  Aerospace  Medicine; 
2012.  Technical  Report  AFRL-SA-WP-TR-20 13-0005. 

12.  Wechsler  D.  Wechsler  Adult  Intelligence  Scale-3rd  Edition  (WAIS-3®).  San  Antonio  (TX): 
Harcourt  Assessment;  1997. 

13.  Powell  DH,  Kaplan  EF,  Whitla  D,  Weintraub  S,  Caitlin  R,  Funkenstein  HH.  MicroCog: 
assessment  of  cognitive  functioning  (version  2.1)  manual.  San  Antonio  (TX):  Psychological 
Corporation,  Harcourt  Brace  &  Co.;  1993. 

14.  Powell  DH.  Profiles  in  cognitive  aging.  Cambridge  (MA):  Harvard  University  Press;  1994. 

15.  Hosmer  DW,  Lemeshow  S.  Applied  logistic  regression,  2nd  ed.  New  York  (NY):  John  Wiley 
&  Sons,  Inc.;  2000. 

16.  Spicer  J.  Making  sense  of  multivariate  data  analysis.  Thousand  Oaks  (CA):  Sage 
Publications;  2005. 

17.  Vittinghoff  E,  McCulloch  CE.  Relaxing  the  rule  of  ten  events  per  variable  in  logistic  and  Cox 
regression.  Am  J  Epidemiol.  2007;  165(6):7 10-7 18. 

18.  Carretta  TR.  Pilot  candidate  selection  method:  still  an  effective  predictor  of  US  Air  Force 
pilot  training  performance.  Aviation  Psychology  and  Applied  Human  Factors.  2011;  1(1):3- 
8. 

19.  Sackett  PR,  Wade  BE.  On  the  feasibility  of  criterion-related  validity:  the  effects  of  range 
restriction  assumptions  on  needed  sample  size.  J  Appl  Psychol.  1983;  68(3):374-381. 

20.  Raju  NS,  Edwards  JE,  LoVerde  MA.  Corrected  fonnulas  for  computing  sample  sizes  under 
indirect  range  restriction.  J  Appl  Psychol.  1985;  70(3):565-566. 

21.  Lawley  DN.  IV. — A  note  on  Karl  Pearson’s  selection  formulae.  Proc  R  Soc  Edinb  A  Math 
Phys  Sci.  1944;  62(l):28-30.  doi:  10.1017/S0080454100006385 

22.  Cohen  J.  The  cost  of  dichotomization.  Appl  Psychol  Meas.  1983;  7(3):249-253. 


13 


DISTRIBUTION  STATEMENT  A.  Approved  for  public  release. 


Cleared,  88PA,  Case  #2015-1652. 


LIST  OF  ABBREVIATIONS  AND  ACRONYMS 


AUC 

area  under  the  curve 

IQT 

Initial  Qualification  Training 

M 

mean 

MAB-II 

Multidimensional  Aptitude  Battery  II 

RPA 

remotely  piloted  aircraft 

SD 

standard  deviation 

USAF 

United  States  Air  Force 
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